1. Field of the Invention
The present invention relates to an apparatus and a method for measuring and converting a 3D shape that can be seen in a plurality of digitalized video images or dynamic images into a 3D model to display it as computer graphics.
2. Description of the Prior Art
It is not possible to use a 3D shape that is a 2D image projected from a 3D body in a plurality of images or video images as an object for computer graphics processing, for lack of information about the length and the orientation of the shape in the 3D space. In order to make it an object for computer graphics processing, a correspondence between the images or between the video images has to be established with accuracy, and the information of 2D images has to be reconstructed as 3D information.
FIG. 1 is a flowchart of a conventional 3D model conversion method. In FIG. 1, 2D data, such as a plurality of images or dynamic images serving as the object to be converted, is given (step S101), and a calibration is performed for each of these images (step S102). Here, xe2x80x9ccalibrationxe2x80x9d means the procedure for determining the viewpoint and the viewing angle, that is, from which direction and distance the images are seen.
Then, a 2D line drawing is generated (step S103), and the correspondence of the faces enclosed by the line drawing is defined (step S104). When the correspondence of each face is clear, a 3D shape and texture can be generated (step S105), and a 3D model is outputted as the result of the specified 3D shape and texture (step S106). An input method for a 3D model using this method is known from Publication of Unexamined Japanese Patent Application (Tokkai) No. Hei 2-219010.
Moreover, as it is usually difficult to establish the correspondence between the faces, methods have been reported at various conferences, in which not the correspondence between faces but between points is designated, and such methods have been developed by many software companies. In methods in which the correspondence between points is designated, a designation of correspondences between points is performed, and it is necessary to calculate their coordinates in the 3D space before the calibration. The methods in which a point correspondence is designated can be roughly categorized in two classes: methods in which the procedure of connecting a plurality of points to designate faces is performed before the calculation of the 3D space, and methods in which this procedure is performed after the calculation of the 3D space.
FIG. 2 is a flowchart for a conventional 3D model conversion method in which the procedure of connecting a plurality of points to designate faces is performed before the calculation of the 3D space. In FIG. 2, 2D data, such as a plurality of images or dynamic images serving as the object to be converted, is given (step S201), and the correspondences between points are designated for each of these images (step S202), and faces are designated by connecting these points (step S203). This operation of designating faces is performed on the image that has been projected into a 2D plane.
Then, after configuring the faces, a 3D shape is generated by calculating the coordinates in the 3D space (step S204), and generating a texture (step S205). Finally, a 3D model is outputted as the result of the specified 3D shape and texture (step S206).
On the other hand, FIG. 3 is a flowchart for a conventional 3D model conversion method in which the procedure of connecting a plurality of points to designate faces is performed after the calculation of the 3D space. In FIG. 3, 2D data, such as a plurality of images or dynamic images serving as the object to be converted, is given (step S301), the correspondence between points is designated for each of these images (step S302), and the coordinates in the 3D space are calculated without configuring the faces (step S303). Now, each point has a 3D coordinate. The faces are designated by connecting these points.
Then, by designating faces by connecting these 3D coordinate points (step S304), a 3D shape is generated, and a texture is generated (step S305). Finally, a 3D model is outputted as the result of the specified 3D shape and texture (step S306).
However, in these methods, when an occlusion occurs between a plurality of images, the problem can occur that the shape of faces in the 2D images differs, and the correspondence between faces cannot be designated or determined correctly. Moreover, even when the correspondence between faces has been designated or determined correctly, there is still the danger that significant errors occur in the determination of the texture of the 3D shape.
Here, xe2x80x9cocclusionxe2x80x9d means the effect that a portion is located behind a face and cannot be seen due to the overlapping of a plurality of designated faces. For example, as shown in FIG. 4, if the portion indicated by bold lines is converted into a 3D model by using two cameras, camera 1 detects a medium-dark portion located between a bright portion and a dark portion, as shown in FIG. 5(a). On the other hand, camera 2 detects only the bright portion and the dark portion, as shown in FIG. 5(b), whereas the medium-dark portion is in the dead angle (occluded) and therefore cannot be detected. In this case, point A rises upward in the detected 3D shape in the view of camera 2 as shown in FIG. 6, or in other words, point A is mistaken for point Axe2x80x2, and a correct shape recognition is not possible.
Moreover, also in the methods for designating the correspondence between points, in the method in which the procedure for designating a face by connecting a plurality of points is performed before the calculation of the 3D space, the arrangement in the 3D space is not yet decided at the time of configuration of the faces, so that the user has to configure the faces while imagining the 3D space, and the more complicated the faces are, the higher is the probability that the face configuration will differ from the real face configuration.
Moreover, in the method in which the procedure for designating a face by connecting a plurality of points is performed after the calculation of the 3D space, a group of points arranged in the 3D space is displayed on a 2D display device, such as a display, so that the more points there are, the more difficult it is for the user to discriminate each point, and the larger is the possibility that a precise face configuration is not possible.
It is an object of the present invention to solve these problems, and to provide a 3D model conversion apparatus and method with which a precise face configuration can be performed even when there is occlusion, and which is easy to operate for the user.
In order to achieve this object, a 3D model conversion apparatus for measuring the shape of a 3D body, based on a plurality of images and converting it into a 3D model, comprises:
an image input portion for reading in a plurality of the images;
an image storing portion for storing the images that have been read in;
a base model memory portion for storing base models expressing reference bodies included in the images;
a base model selecting portion for selecting a base model defining a geometric quality of a body in the images, which is to be converted;
an image calibration portion for performing a plurality of image calibrations based on the selected base model, and superimposing the base model on the images to determine 3D coordinates of vertices of the base model;
a face configuration and corresponding points designating portion for designating at least one new vertex and configuring a surface, based on the result of the image calibrations and the previously determined 3D coordinates of the vertices of the base model;
a 3D information calculation portion for calculating 3D positional information of the new vertices when the new vertices are being designated;
a 3D shape display portion for displaying a 3D model of the body, which is to be converted;
a texture image generating portion for generating a texture image to be pasted on a surface of the 3D model, based on a correspondence relation between the 3D model that is finally obtained by addition of the new vertices and the images;
a generated model output portion for outputting data of the completed 3D model.
With this configuration, the user can ascertain visually the state of the 3D model that is being generated, so that a 3D model can be constructed as intended by the user, and a faulty recognition of the 3D model due to occlusion can be avoided.
Next, to achieve the above-noted object, a computer-readable recording medium stores a program for execution on a computer for measuring the shape of a 3D body based on a plurality of images and converting it into a 3D model. The program includes:
reading in a plurality of the images, and storing the images that have been read in;
storing base models expressing reference bodies included in the images;
selecting a base model defining a geometric quality of a body in the images, which is to be converted;
performing a plurality of image calibrations based on the selected base model, and superimposing the base model on the images to determine 3D coordinates of vertices of the base model;
designating at least one new vertex and configuring a surface, based on the result of the image calibrations and the previously determined 3D coordinates of the vertices of the base model, and simultaneously calculating 3D positional information of the new vertices and displaying a 3D model of the body, which is to be converted;
generating a texture image to be pasted on a surface of the 3D model, based on a correspondence relation between the 3D model that is finally obtained by addition of the new vertices and the images; and
outputting data of the completed 3D model.
With this configuration, by loading and executing this program on a computer, the user can ascertain visually the state of the 3D model that is being generated, so that a 3D model conversion apparatus can be realized, with which a 3D model can be constructed as intended by the user, and a faulty recognition of the 3D model due to occlusion can be avoided.